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PREFACE 



Who wants to know about the impact of education programs upon the children and 
youth of our communities, states, and nation? Who has the answers to such 
questions as these: Is my child improving in read^.ng? Are our youth prepared 
for finding and holding down jobs? Do young children with exceptional needs 
benefit from early school programs? Is it possible to evaluate special educa- 
tion programs? 

The California State Department of Education evaluates publicly supported 
educational programs and assists local educational agencies, parents, and 
communities in developing Che capability of improving their programs through 
timely, verified, helpful program evaluation. 

As the California Master Plan for Special Education was being develop.ed, the 
need for special education program evaluation was translated into action. 
Project SEEM (Special Education Evaluation Models for California) was created 
by the California State Department of Education, and it was funded under the 
Education of the Handicapped Act, Title VI-B, Public Law 91-230,, from July 1, 
1972, to June 30, 1975. 

Under the direction of Margaret Scheffelin, Project SEEM has acted as a cata- 
lyst in bringing about cooperative interchange between program evaluation, 
special education, and state and local levels of education. In April, 1974, 
the Operational Project Audit Team recommended to the Department that the 
third and final year of Project SEEM be singularly devoted to the production 
of a document describing critical elements which must be considered in evalu- 
ating special education programs. This publication. Program Evaluation and 
Pupil Information in Special Education , is the result of the audit team's 
recommendation. 

This document is a departure from the usual publications of the State Depart- 
ment of Education. Typical consumers of Department publications may approach 
their reading of this document looking for guidelines, procedures, and direc- 
tives. It is the intent of this document to stimulate, not to prescribe or 
proscribe. Following the recommendations of the audit team, the final chapter 
contains seminar questions related to the models presented in the body of the 
document. The Consumer Model for Program Evaluation and the Synthesis of 
Abilities Model must be considered together, although they may be studied 
separately. 

Appreciation is extended to the many persons throughout California and the 
nation who were involved in Project SEEM activities throughout the three 
years of its operation. Special appreciation is extended to Russell Forney 
and Mary €ampasano, of the Department, and to the many persons who reviewed 
earlier drafts of this document. 
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As we press on with the exciting challenge of working and living with children, 
we find the formula for improving the nature of the human race-total commit- 
ment to the education of all our children. The key questions ^^^^^^J^; 
Are pupil abilities discovered, enhanced, and maintained as a result of what 
we do? How do we know? 



DONALD R. McKINLEY 
Chief Deputy Superintendent 
of Public Instruction 



ALEXANDER I . LAW 
Chief, Office of Program 
Evaluation and Research 



J, VINCENT MADDEN 
Assistant Chief, Office of 
Program Evaluation and Research 
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CHAPTER 1 

Introduction to Program Evaluation in Special Education 



Purpose and Overviev 

Special education program leaders are now gearing up to provide accurate ) 
understandable y and timely information to decision makers, including 
themselves* Information about pupil performance and change can be used 
to meet the requirement of accountability as well as to improve programs 
for children and youth of all ages who require special education* 



This document is designed to stimulate special education program leaders 
at all levels of education in improving their programs through program 
evaluation and pupil measurement* The document is intended to be universal 
and to apply to all education programs enrolling special education pupils> 
regardless of location or legal authorization* The document is not 
intended to be an operational manual* 

This document contains five chapters covering program evaluation and 
pupil information and presenting two new models for discussion and adap- 
tation for use in education programs* The new models are the Cohsumer 
Model for Program Evaluation and th^ Synthesis of Abilities Model for Pupil 
Information. £ach model has the pupil as the focal point* The final 
chapter contains suggested seminar questions for use by discussioa groups* 



Program Evaluation is the process of agreeing to, describing, measuring, 
reporting, and Judging those elements which affect management decisions* 
In education the management begins with the teacher and includes all the 
school staff as well as the family or guardian* In special c^ducatlon the 
^'management** often includes professionals in health fields and other disci- 
plines such as engineering. Evaluation links together goals, needs, 
objectives and outcomes by providing the answers to critical management 
questions, chief among which is: Who wants to know what? 

Description of pupil performance and measurement of pupil change are the 
chief obstacles to program evaluation in special education. Deviation 
(differences between pupils as a group) and variability (differences 
within individual pupils) are the characteristics which create measurement 
and reporting problems for special education program leaders and evaluators* 
Focus on pupil abilities over a period of time appears to be the one way of 
dealing with the problems of deviation and variability* Direct measures of 
elements of pupil performance in educationally relevant tasks are recon?nended 
in the three phases of learning: preparation, acquisition, and use* Learn- 
ing is defined as a measurable, verifiable change in performance* 

, ■ s 
1 ■ • •• 



LiCerature Review 

There exist three bodies of literature bearing on the topic of program 
evaluation in special education. The three are regular education program 
evaluation, special eci^cation, and special education program evaluation. 

Each of these bodies represents a different set of professionals with 
their own experiences and reporting to their own constituents in their 
own terms. 

Regular education program evaluation literature on designs and measures 
has not been found directly useful to special education programs. Two 
works, however, have been helpful in the development of this document, 
Stake (1967) and the California State Department of Education (197A). 
Stake emphasizes the importance of description and judgment. The Depart- 
ment of Education's handbook emphasizes the multiple uses of pupil and 
program Information in educational program planning, operation, evaluation, 
and reporting. 

Special education literature contains an immense number of publications 
in the area of pupil assessment and pupil measurement. All of the publi- 
cations put together have been of no use in conceptualizing a framework 
for special education program evaluation and pupil measurement to encompass 
Che range of ages (from 0-21 years) and the range of pupil characteristics 
in special education programs. In addition, the mass of technical terms ^ 
serves to confuse or to Infuriate the reader or listener. (Scheffelin, 1969) 

The special education orogram evaluation literature is recent, sparse, and 
marked more by description of .the problems rather than by suggested solutions 
or approaches. The approaches are primarily aimed at short-term new projects 
rather than at continuing programs. Examples of approaches are Meierhenry, 
(1969) and Gallagher, Surles, and Hayes, (1973). Examples of problems are 
Prager (1971), Jones (1973), and Nomos (197A). 

In all three bodies of literature, evaluation has been treated as an end to 
itself, not as a part of a whole cyclic process. More attention has been^ 
given to the technical aspects of designing and conducting evaluation studies 
than to the information-using consumers of evaluative information. 

Background 

Program evaluation and pupil description In special education have taken 
different forms throughout Its history. The different forms can be 
considered as protest, testimonials, and testimony. All forms can be 
occurring at the same time. Program evaluation began with the pleas of 
parents before local and state boards and legislators to obtain programs 
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for their particular youngsters for whom no program existedv Program 
evaluation in special education then moved to the testimonials of parents 
as they sought Continuance and expansion of those programs. Too often 
parents were so pleased at the provision of a program, any program, that 
they did not press on for an improved program. New parents accepted the 
program as it was and until recently, tended not to decaand program 
improvement. Program evaluation has taken the form of testimony in courts 
as parents sought access to appropriate programs or egress from and non-* 
entrance into inappropriate programs. These activities were, by their 
nature, sporadic and aimed at immediate change in education programs. 
Program evaluation in special education is moving past the use of tests 
and test scores as indicators of program quality and pupil success. 



Pupil description has taken different forms also as parents and profes- 
sionals attempt to provide appropriate programs for individual pupils. 
Technical terms go out of favor and drop from usage; some by comsion consent 
others by consent decree. Semantic overlays creep in on any term as pupil 
characteristics begin to be attached to the term. More terms exist in the 
assessment literature than in the instructiv nal literature. Assessment 
terms usually need interpretation and translation to nonprofessionals. 
The advent of due process procedures is a move toward laicizing educational 
decisions and educational terminology. 



Statement of the Problem 

Program evaluation and pupil description in special education must move 
from external, intermittent, unbudgeted, and disability-specific to 
internal, continual, funded, and pupil performance-generic. \ 



Principles of Program Evaluation in Education 

The following statements have been generated from the results of several 
years of working intensively with evaluation and program staff in a variety 
of special educational programs at school, district, state, and federal 
levels. Program managers have successfully adopted these principles as 
fundamental assumptions in planning, conducting, and using the results of 
program evaluation* The principles are pertinent to all education programs 

1. Evaluation in education is the process of agreeing to, describing, 
measuring, reporting, and judging those elements in education which 
affect management decisions on program improvement at pupil f local, 
state, and federal levels. 

2. The purpose of program evaluation in education is to improve, not to 
prove or to reprimand. 

3* Evaluation is an integral aspect of planning, operating, and modifying 
programs. Evaluation and planning are inseparable. 
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4. Variability in the performance of pupils poses measurement and reporting 
problems for program and evaluation staffs, but that variability does 
not exempt education programs from the requirement of accountability. 

5. An updated program description, including comparison of old baseline 
pupil data with new data, is an essential form of program evaluation. 

6. The test of a program description is; Is it described in operational 
terms sufficient to permit on-site verification and judgment about 
process and outcomes? 

7. It is an indefensible waste of resources to gather, analyze, interpret 
and report unreliable data or to require data gathering without a 
plan for its use. 

8. In human beings learning, growth, and development are never straight 
lines. They are sporadic rather than continuous; they go down as well 
as up. 

9 Useful research information is a product of carefully described program 
operations. Research can be most useful to the researcher and to 
others when the research is designed by and with program staff to yield 
timely and accurate information on specific program elements. Sound 
program evaluation designs point up individual pupil effects which 
would have been hidden by averaging group results. 

10. Measures of low reliability or unknown reliability, whether tests check- 
lists, scales, or inventories, yield little useful information, ^et that 
unreliable information is often secured by, or provided to, incompletely 
trained staff, with an unknown effect on pupil assessment, assignment, 
instruction, and reassignment. It is better to train observers than to 
increase the number of unreliable measures. 

11. Informed judgment can yield reliable data about pupils and programs, 
given a consensus on standards and a set of standard procedures for 
exercising judgment. 
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CHAPTER II 
The Consumer Model for Program Evaluation 



This chapter discusses consumers of evaluation and presents the consumer 
model of program evaluation developed for this document. 



Consumers of Program Evaluation 

As alluded to earlier in the preface, the prpgrain evaluation literature 
has emphasized the procedures and the doers of evaluation rather than the 
consumers of evaluation and evaluative information. In the model presented 
in this docuaent, all persons in the governance structure of education are 
considered consumers. Figure 1 presents the chief elements of the gover- 
nance structure of elementary and secondary education. The elements are 
the pupil, parents, local school, school district , school board, chief state 
school officer, state board, governor, Legislature, the President and the 
United States Office of Education, the Congress, and the courts. The inter- 
related nature of the governance structure is emphasized in the schematic 
chosen for its portrayal. The pupil is at the center of the schematic, 
signifying the point of interaction of the elements as well as the reason 
for the existence of the educational elements of the governance structure. 



It should be noted that although institutions of higher education are 
important to education and particularly important in special education, 
they are purposely omitted from this discussion. To incl\ide them would be 
beyond the scope of this document, which is aimed at the public schools. 

TWo questions should be asked about the consumers: (1) Who are they? 
(2) Under what conditions will the consumers use program evaluation? 
These questions are answered below in a general way* A program leader 
must discover particular consumers' requirements for evaluation and infor- 
mation. A program leader should obtain specific questions which consumers 
have about the programs, and» what is equally important, the kind of evidence 
consumers will accept as answers to their questions. 



A. People use program evaluation, acting individually, or collectively 
as members of a group. Five roles are distinguishable: (1) program 
stafiE, such as teachers, administrators, support staff, at local, 
state, and federal levels; (2) parents or guardians of the pupils; 

(3) community representatives, such as employers of pupils and grad- 
uates, governors, legislatures, fimding agencies, boards of education; 

(4) pupils, enrolled in special education programs, or graduates, or 
former enrollees; and (5) evaluators, working with or for other 
consumers. 

12 
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B. Consumers, that is, people and institutionsV will use program evaluation 
and continue to use it, if and only if certain conditious are met: 

1, It's helpful. 

2, It's done in time. 

3, They helped design it so it gives them answers to questions they 
asked, in language they understand, 

4, Th«?y know where the numbers come from. 

5, They trust the evaluators . 

6, They agree with the purpose; program improvement for improved 
pupil performance. 

7/ Any external evaluation is independent of the administrative 
supervision of the program and is free from conflicts of 
interest • 

8^ Any internal evaluation is negotiated by the administrative 
supervision of the program with the program operators . 

9^ They know it's not busywork because it grows out of program 
operation (although they know it's still work) . 

10, Money and time are available, and help is there when they 
need help. 



jsa^pn rial Features of the Model 

Eight essential features are the building blocks of the consumer model for 
program evaluation developed for this document. As shown in F^Sure 2, 
heading from the bottom to the top, the eight features are responsibility, 
budget, trained people, reporting, use of information, design, measures, 
and basis for evaluation. The model is applicable to all levels of educa- 
tion and in all elements of the governance structure. 

Education program leaders and other persons in the governance system can 
use the consumer model presented in this chapter as a means of discussing 
their expectancies for program evaluation. 

Bxtmoles of the use of the consumer model are given ia Figure 3. Here the 
«i88ential features are applied to two elements of the governance structure, 
the local school and the Legislature. Note the reciprocal relationships 
throughout, typified by the reporting feature. The Local School element 
Biust provide concise, understandable reports, keyed to consumer requests 

13 
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FIGURE 1 

SCHEMATIC OF THE GOVERNANCE STRUCTURE 
OF EDUCATION 
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FIGURE 2 

ESSENTIAL FEATURES OF THE CONSUMER MODEL 
FOR PROGRAM EVALUATION 
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FIGURE 3 



APPLICATION OF THE CONSUMER MODEL 
FOR PROGRAM EVALUATION 



ESSENTIAL 
FEATURES 



Responsibility 



Budget 



Trained People 



Reporting 



LOCAL 
SCHOOL 



LEGISUTURE 



Use of Information 



Design 



Measures 



Basis for 
Evaluation 



Provide accurate and timely information to 
consumers including themselves 



An integral part of the program budget 
with discernible time, facilities and 
resources 

People who have learned to 

- look at and listen to pupils 
• separate data from inference 

- record, retrieve, and communicate pupil 

information 
People who accept pupil measurement as 
their responsiblity 



Concise 

Understandable 

Kifiv! to consumer requests 

Related to program objectives 



Modify school programs to fit changing 

pupil abilities. 

Report to other consumers. 

Related to pupil abilities and program 
objectives. 

Created with the people who know the 
pupils. 

Fiipil performance in terms of abilities 
in educationally related tasks. 



Description of education program. 
Heasurenent of pupil change In abilities 



Provide useful and timely feedback 
to providers and consumers 



Recognition that program evaluation 
is continuous and Is a necessary 
part of program operation and 
management 

People and institutions who have 
learned to accept both positive and 
negative -results as useful results. 
People who accept the fact that 
human change is not linear. 



Explicit, answerable questions. 
Resources supplied to secure adequate 
data to arrive at conclusions. 



Judge school programs. 

Agree to demonstrated need for 

revisions. 

Supply required resources. 

Agreed to in advance of program 
operation. 



Consensus on validity of measures 
and acceptability of evidence. 



Description of progm results in 
terms of pupil abilities. 
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and related to program objectiveg. The Legislature element must provide 
explicit, answerable questions and supply resources to secure adequate 
data to arrive at conclusions. 



Education program leaders can use the features of the model as the basis 
for staff self*study of their current program evaluation efforts. Program 
leaders who discover strong points in the program evaluation efforts can 
be justifiably proud. Chances are, though, that most programs will have 
one or more weak areas. A rule of thumb is, "If it's not^ written down» 
and handy, it's probably not being done." Information and procedures in 
peoples' heads usually stay there and are not systematically passed on. 
A leader should inqnire about these points, not only with the program 
staff but also with other people inside and outside of the school. 



Chances are that little or no reliable, verified baseline data are avail- 
able in terms of pupil abilities and perfonoance on educationally related 
tasks. If so, the education program leader must shift emphasis from 
program evaluation to program description and must gather educationally 
relevant pupil data. The Synthesis of Abilities Model presented in 
Chapter Four will provide a simplified framework for thinking about and 
organizing pupil data. 
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CHAPTER III 



Pupil Information in Special Education 

Education exists to enhance the performance of pupils. Schools are 
accountable to education consumers for accurate and timely reports of 
the impact of education programs upon their pupils. Education leaders 
are searching for new ways of describing pupils and presenting pupil 
information. 



Statement of the Problem 

In the absence of a unifying framework for organizing pupil performance 
information, education program staff tended to remain in their areas of 
professional experience. They retained their professional disciplines' 
terms and the ways of thinking about pupils represented by their disci- 
plines. Pupil description has been discipline or categorically specific, 
and primarily deficit-oriented. 

Few human beings make a living by exhibiting their performances in their 
deficit areas, a fact well understood by employers. Pupils, parents, 
school staff, and the comminity want to know what pupils can do, their 
abilities, along with clear and understandable descriptions of what the 
pupils can't do, their deficits. Comminication within the governance 
structure is made more difficult when no comoon set of terms exists for 
describing pupils and reporting the results of measuring pupil change. 
Even when a conmon set of terms exists, the terms haven't always meant 
the same things to different members of the governance structure. 

Education consumers are not satisfied with the pupil information currently 
available. Measurement and reporting of program iiiq>act must move from 
eligibility and attendance data to information about pupil performance and 
change. The deviation and variability of special education pupils must be 
taken into account in the development and use of new pupil information 
structures. 



Background 

Historically, two types of pupil information were collected by schools, 
eligibility and attendance. Attendance information was reported; eligi- 
bility information was not. Eligibility was presumed from the reporting 
of attendance. 
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Neicher eligibility nor attendance was necessarily pupil performance 
information. Eligibility has related to individual pupils. Attendance 
data has been considered an aggregate for programs. 



Eligibility information has been used primarily to make decisions on 
the enrollment of individual pupils into existing educational programs, 
regular and special. Attendance information has been used to make 
decisions on the amount of money to be requested, received, and expended 
by the program administrators at the various levels of education: local, 
state, national. 

Eligibility information required for enrollment in regular education 
programs has typically included three factors: age, ability to profit 
from instruction, and absence of harm or danger to other pupils. Each 
of the three factors has required a different type of evidence. For 
age, written proof of birth or existence has been required. For ability 
to profit from instruction , school staff judgment has typically been 
exercised at the building site level. For absence of harm or danger W 
other pupils , school staff judgment or medical opinion has typically 
been required. 

In special education, eligibility information has been necessary for 
enrolling pupils. Eligibility information was of various kinds, obtained 
by a variety of methods, and described in a variety of terms and concep- 
tual frameworks. Most eligibility information has not been directly 
related to pupil performance in educationally relevant activities. 
Eligibility information was not usually translatable into instructional 
information. The measures used to determine eligibility typically did 
not lend themselves to measuring changes in pupil performance, especially 
over long periods of time or developmental phases. Furthermore, certain 
measures had little direct relationship to education, although the conditions 
they measured had a great deal to do with the educational need of the pupil. 
Many measures of eligibility had little to do with the instructional objec- 
tives for an individual pupil or with the goals and objectives of a 
special education program. 



Enrollment procedures for regular and special education programs, contained 
in a variety of written laws and uncodified practices, continue to be the 
subject of legislation and litigation. Litigation continues across the 
country on behalf of persons excluded by law, public policy, or local 
practice from eligibility for enrollment in any form of public education. 

Attendance information has typically included time spent on the school site 
or time in directly supervised instruction off-site. A bewildering array 
of time periods and conversion formulas exists across the nation. Attendance 
information has not been systematically used as one basis for program plan- 
ning and program evaluation. 
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Progrsm Impact information has been generated and reported in a variety of 
formats* The most typical pattern has been an individual pupil report card 
or conference with the parent and a district or state-vide aggregation of 
test score data for large numbers of the pupils tested. It has become 
increasingly apparent that no segment of society willingly accepts permanent 
custody of the lowest achievement levels. 

No single measure, no single dimension could adequately convey the complex 
information about infants, children, and youth, ages zero through twenty-one, 
with a wide range in all aspects of growth and development. Examples of 
incomplete dimensions were age, developmental level, intelligence, height, 
weight, achievement, and sensory capacity. People didn't arrange themselves 
neatly and linearly. Attempts to force a linear progression led to artificial 
arrangements in which events are allegedly linked to numbers, but interpre- 
tation of the numbers was risky because numbers didn't know where they came 
from. The danger arose because numbers, once stated, seemed ^^o be reified 
and took on a life of their own. 



In the absence of reliable and valid measures of pupil performance and change 
in the areas of educational emphasis, program staff tended to adopt one of 
three possible courses of action. One course was to use inappropriate or low- 
power measures, ignoring (or not aware of) the limitations on interpreting 
the results. Another course was to create and use measures of unknown reli- 
ability. Either way, the "numbers didn't know where they came from." The 
third course of action was to avoid measurement. All three courses of action 
led to little or no possibility of demonstrating and verifying program impact 
on pupils. 

Principles for a Structure of Pupil Information 

The following statements have been developed by Project SEEM from its 
experiences in working with consumers of pupil information. These statements 
have been used as criteria for the development of a new structure for organ- 
izing pupil infonnation, presented in Chapter IV. The prloAry emphasis in 
the statements is on the consumers of pupil information* As shown in Chapter 
II, the consumers begin with the pupil and extend to the entire governance 
structure of education, including the courts. 

To be useful to all consumers, a structure for organizing pupil information 
must meet all these criteria; 

1. Enhance conminication among all consumers of education and program 
evaluation. 

2. Emphasize the abilities of pupils. 

3. Be applicable to all pupil study procedures from screening and referral 
to exit and follow-up. 
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4. Enable zneasurement of current pupil performance* 

5. Increase the descriptive power of existing pupil measurement procedures. 

6. Assist in selecting instructional areas of emphasis, 

7. Promote the setting of feasible instructional objectives. 

8. Be useful in predicting the time period for the attainment of pupil 
objectives in the selected curriculum. 

9. finable the measurement and reporting of pupil change. 

10. Be verifiable by trained observers and listeners, 

11. Provide the basis for documentation of pupil progress. 

12. Promote maximum use of existing pupil information. 

13. Be usable by all professional disciplines related to education. 

14. Be universal in its applicability to all pupils in all instructional 
settings. 

15. Recognize individual differences and capitalize on deviation and 
variability. 

16. Recognize the role of neurophysiology in human action, 

17. Emphasize the essential focus of education on instruction and assessment, 
and the focus of the pupil on learning and performance, 

18. Be aggregatable for program planning, operation, evaluation, and revision. 

19. Be parsimonious: sufficient, consistent, and efficient. 

These principles can be used by readers of this document, education program 
staff and other consumers in studying the new Synthesis of Abilities Model 
presented in Chapter IV. 
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CHAPTER IV 
Synthesis of Abilities Model for Pupil Information 



It is the purpose of this chapter to present a new and simplified structure 
for organizing pupil information, the Synthesis of Abilities Model. The 
model emphasizes the pupil's ability to learn, introduces the notion of 
pupil performance elements in the three phases of learning as a unit of 
analysis in education, and highlights trained human judgment as a measuring 
instrument. 



Delimitations of the Model 

The Synthesis of Abilities Model takes into account the distinctions between 
the actions of human performance and external observations of that human 
performance. 

The distinctions are shown in the universally used modal verb forms. In 
English, can, will, and do are the modals of chief interest in describing 
human performance in the Synthesis of Abilities Model. (It is recognized 
that additional modals are applied to human performance. Examples are max. 
might , should , would . ought .) 



Every human action is a blend of can and will . The can stems from the human 
body, the will stems from the human person. Both can and will are involved 
in human performance. The external observer observes an action as a do, 
whet:her the observer is a person or a calibratable instrument such as a 
grip-strength meter. 



The modal verbs can, will .and^ do are critically important in at least two 
ways. First, the modal verbs convey a sense of choices open to the actor, 
the person performing the action. In fact, the choice may not be open in 
all cases. For example, *'hand-preference" is a two-choice situation only 
for people who have two hands. Second, the negative forms of those three 
modal verbs can't , doesn't , and won' t , convey a complex set of meanings to 
the reader or hearer. For example, doesn't is interpreted as can't by one 
observer and as won't by another observer. Those differing interpretations 
may make large differences in the conclusions reached and the recommendations 
offered by the original observers. The differences can easily be magnified, 
by time and distance, to other receivers of oral or written reports who do 
not have access to the primary obsexrvation data. 
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The Synthesis of Abilities Model is focussed on the pupil and on information 
about the pupil's abilities as shown by the pupil's performance. The pupil 
is considered in the role of an acting and responding human being who trans- 
duces energy. The model is limited to those aspects of a pupil's life which 
are public and verifiable. This is not to deny that a private world exists 
of intention and reflection. But enforcing self-report from the pupil is 
tantamount to an invasion of privacy. 



Structure of the Synthesis of Abilities Model 

An overview of the Synthesis of Abilities Model is displayed in Figure 4. 
There are four main divisions: Ability, Sphere of Performance , Analysis, 
and Synthesis. 

Ability is the first division of the structure of the Synthesis of Abilities 
Model. Ten abilities have been identified. They are the ability to 

(1) transmute substances; (2) transduce energy; (3) process sensation; 

(A) control body movement; (5) process symbolic information; (6) relate; 

(7) communicate; (8) survive; (9) hold one's own as a peer; and (10) learn. 



Sphere of Performance is the second division of the structure of the 
Synthesis of Abilities Model. Five spheres within which pupils act and 
respond have been identified in Figure 4. The spheres are (1) the internal 
environment; (2) the external environment; (3) other people; (4) society; 
and (5) instruction. The spheres overlap and are not mutually exclusive. 

Analysis is the third division of the structure of the Synthesis of Abilities 
Model. Two types of analyses are included, quantitative and contrasLiye. 
Each type of analysis is applied to information about pupil abilities in all 
spheres of performance. Quantitative analyses are done in measuring or coun 
ing the critical elements of a pupil's performance. Contrastive analyses 
are done in comparing the measurements with previous measurements on the 
same pupil or with other measurements of other pupils. 



Synthesis is the fourth division of the structure of the Synthesis of 
Abilities Mo4el. Synthesis includes the aggregation . and integration of 
the analyzed information on pupil abilities from the performance elements 
in each of the spheres of performance. Synthesis prevents isolation of 
information and omission of important areas of a pupil's life. Synthesis 
promotes the communication and reporting of changes in pupil abilities to 
all interested parts of the governance structure of education. 



The model will be discussed in detail in three sections: (1) Abilities 
and the Five Spheres of Performance; (2) Analysis; and (3) Synthesis. 
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FIGURE 4 
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Section It Abilities and the Five Spheres of Perfomance 

Each of the ten abilities has been assigned to one of the five spheres of 
performance* 

Sphere I, The internal environment * Two abilities have been identified; 
the ability to transmute substances and the ability to tr ansduce energy* 

Sphere I, the internal environment, is within the human body* In this 
document, the pupil is not considered a learning machine* Rather, the 
pupil is considered as a human being who transduces energy in time and 
space* The specifications for the human body and its neurochemical 
working are incompletely known* However, lack of complete knowledge 
should not be allowed to prohibit the use of the partial knowledge accu- 
mulated over the years in neurophysiology* 

All human actions depend on energy generated within the body* To generate 
energy, the human body transmutes substances taken into it* Examples are . 
the complex interactions between the endocrine glands, the digestion of 
food, and the use of oxygen to ignite carbon* (How many persons have ever 
thought of themselves as miniature chemical laboratories and furnaces?) 

To keep alive or to maintain growth and development, the human body must 
transfer energy within itself* Examples of the energy transferred are the 
electrical impulses of the central nervous system and the hydraulic pumping 
of the circulatory systems* 

Performances in Sphere I are internal actions, functions, and/or reactions* 
Examples are typically physiological, such as temperature, sleep, electrical 
activity, or joint extension* Sphere I is involved in studies pursuing the 
educationally important interactions between internal environmental sub- 
stances such as medication and Spherell abilities such as moving body parts* 

Sphere II, The external environment * Three abilities have been identified: 
the ability to process sensation , the ability to control bodv movement, and 
the ability to process symbolic information * 

Sphere II, the external environment, is outside of the human body* The 
physical environment includes energy, objects, space and time* In this 
document, the pupil i8 not considered to be a responding machine whose every 
action is controlled by external factors. Rather, the pupil is considered 
as a human being who tranduces energy, takes up space, acts on objects, and 
exists during time* 
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Each of these aspects of the external environnent is dif ferentiable. Energy 
is defined as light, sound, pressure, heat, and aromae Objects are defined 
as countable things having contiguous surfaces and mass, such as trees, 
air, cars, pencils, dogs. Space is defined as the field in which objects 
rest or move. Tine is defined as the succession of cyclical or predictable 
events. 

The ability to process sensation includes the functions of seeing, hearing, 
feeling touch, feeling movement, feeling temperature, smelling, tasting, 
knowing where one is in space, feeling change in location. 

The ability to control body movement includes the entire musculature under 
the control of the human being. Once there was thought to be two separate 
systems, one "autonomic," not subject to voluntary control, and one "vol- 
untary." The separateness has never been complete, and evidence on the 
extent of voluntary control over "autonomic" systems is accumulating and 
gaining acceptance. Muscles make the body parts move, literally, from 
head to toe. Muscles also make the movable body parts not move. Control 
of a myriad of muscles enables the person to make vocal sounds, snap fingers, 
hold a pencil or paintbrush, chew or swallow food, breathe, focus the eyes, 
turn a somersault, swim, sit, and smile; in short, every performing verb in 
the dictionary. All actions involve muscles. 

The ability to process symbolic information includes a series of subabilities: 
coding, storing, associating, integrating, retrieving, and expressing parts 
or patterns of the external environment. 



Symbolic information is available in all the forms of objects in Sphere II, 
the external environment. Symbols are objects or patterns of objects which 
stand for other objects, events, or patterns of events. Energy as an attri** 
bute of all objects is an intrinsic part of Sphere II. There are naturally 
occurring signs such as the tracks of animals, the sounds of dropping water; 
and star patterns. There are deliberate signs made by human beings such as 
smoke patterns, pictographs, traffic lights, spoken words, gestures, hand* 
shakes, musical patterns, printed numerals, written documents, and whistling 
sounds. 



Coding begins with the processing of sensation. Expressing ends with the 
controlling of body movement. Between coding and expressing are events not 
yet understood. These events take place in an internal environment not yet 
susceptible to explication, and certainly not ethically open to deliberate 
experimentation in human beings. 

The six subabilities postulated for this document have been identified from 
the inferences of its writer from direct observation of persons engaged in 
processing symbolic information. These subabilities have also been identiiried 
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by researchers in huaisn perforfflsnce in learning aad ■emory. It is note- 
worthy that the conceptualisation offered here extends to creativity, which 
can be considered reorganising reality, repatteming the envirooMnt, or 
focussing on a new and different part of the environment. The governance 
structure of education continually draws attention to pupil performance in 
two areas of symbolic infoCTMition, language and mathematics. Processing 
visual symbols for auditory language is involved in both reading and 
arithmetic. 

Sphere III. Other people . Two abilities have been identified: the ability 
to relate and the ability to conmnicate . 

Sphere III, other people, is outside of Sphere I, the internal environment, 
but coexists with Sphere II, the external environment. The role of other 
people in the life of a pupil is so important that Sphere III i» * necessarily 
separate sphere in this document. Other people are considered in the pupil s 
person-to-person relationahips with them. These relationships are carried 
on in two modes: bodily presence and voice. 

The ability to relate includes the functions of getting sion/j with other 
individuals in the same proximity, meeting new people, and perceiving the 
feelings or the intentions of other individuals. 



The ability to comunicate includes the functions of receiving «nd sending 
messages. 

Sphere IV. Society . Two abilities have been identified: ^-ha ability to 
survive and the ability to hold one's own as a peer * 



Sphere IV, society, is outside of Sphere I, the internal environment, but 
coexists with Sphere II, the external environment and Sphere III. other 
people. Society is considered as the rules and patterns of organisation 
are created or maintained by people acting together. Society includes 
historically natural units such as families and cultures. Society, for 
purposes of this document, includes regularly occurring units such as school 
classrooms, restaurants, business offices, bus passenger loads, civic com- 
mittees, hospital wards, factories, and political governing bodies. 

Just as Sphere IV is an abstraction or generalisation of spheres I, ii. III, 
so the abilities in Sphere IV are abstractions, referenced to the expecta- 
tions of society for one's peers. Exhibiting patterns of abilities in 
spheres I, II, III, and IV appears to qualify a hum«n being as possessing 
one or more abilities in Sphere IV. The ability to hold one's own as a peer 
presupposes the ability to survive. The exact conditions for survival 
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depend on the constellation of abilities possessed and the requirements of 
the particular facet of society involved. Suffice it to say that no human 
being has the ability to hold one's own as a peer in all possible facets 
of society under all possible forms of the external environment and with 
all other people. 

Sphere V> Instruction ^ One ability has been identified: the ability to 
learn * 

Sphere V, instruction, permeates the four other spheres. Instruction takes 
place in a variety of environments, with a variety of people, and in a 
variety of instructional modes. Although education's chief emphasis is on 
learning, educators recognize that they are not the only instructors of any 
pupil. Parents, family members, peers, people in the community, and the 
pupil assist in the learning process. 

Learning is defined as a measurable, verifiable change in performance. 
Three subabilities have been identified for this document; preparation, 
acquisition, and use. Each subability can be considered a phase of 
learning. In each phase the pupil performs. The particular actions per- 
formed by the pupil are different according to the task being performed. 
However, certain elements of performance are the same in all tasks in the 
three phases of learning. For example, every task takes place in time and 
is composed of actions. 



Section 2; Analysis 

Two types of analysis have been identified: quantitative and contrastive. 
Both types of analysis apply to all ten abilities in all five spheres of 
performance. Each type of analysis will be discussed separately. Because 
the emphasis of education consumers is on Sphere V, Instruction, more 
attention will be given to analyses of performance elements in the three 
phases of learning. 



Quantitative Analysis . The units of analysis for each ability and sub* 
ability in spheres I, II, III, and IV depend on the particular task per- 
formed and the level of data collection. The level of data collection 
depends on the reason for pupil study, the discipline collecting the data, 
and the evidence required by the consumer. The types of data vary from 
blood chemistry information to speech production to entire life history. 



Persons assessing pupil performance in spheres I, II, III, and IV are 
typically assessing the use of an ability. For exampie, in Sphere II, 
processing sensation, one of the subabilities is detecting sound. The 
person assessing the ability of a child to detect sounds presents a series 
of discrimination tasks to the child. Typically, the child indicates the 
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presence of sound by performing a particular action and indicates the 
absence of sound by not performing the particuUr action. The person 
assessing the child's ability is prepared to switch to an instructional 
mode in which the child learns to perfofa the task of indicating whether 
or not a sound is present. The indication task is a form of self-report. 

Persons assessing performance in Sphere y are assessing elements of pupil 
performance in the three phases of leartjing: preparation, acquisition, 
and use. Examples of performance elements are the aunber of opportunities 
for responding, the amount of support required by the pupil from other 
people, the number and types of demonsttations required, the consistency 
of the actions performed, latency (the time per^*"* between the signal to 
act and the pupil's action), the energy required to perform the task, the 
appropriateness of the action, the language of any verbal directions. 

caution must be exercised at this point. Little information on these 
Sphere V elements of performance is generally J Uble on anjr population 
or on any contrast referent. In addition, special problems exist, such as 
determining and measuring differences and specifying the evidence^ required 
for verifying both the measurements and the differences. It is clear that ^ 
persons working in Sphere V must acquire measurement skills for dealing with 
variability and reporting skills for dealing deviation. Variability 

is defined as differences within Individual pupils. Deviation Is defined 
as differences between pupils as a group, use of performance elements as ^ 
the units of data collection and analysis perffli" Information to be reported 
to consumers, provided there has been agreement on the performance elements 
before the data collection begins. 

Contrastlve Analyses . A quantitative description of pupil performance 
elements In each sphere of performance u placed In a frame of reference 
by comparing the description with one or more other descriptions of ^per- 
formance. The other descriptions are chosen fro» four frames of reference, 
or contrast referents: typical persons or atypical persons, as groups or 
as Individuals. 

The pupil herself or himself Is a special case of an individual contrast 
referent. The accumulation of performance element information is an Indi- 
vidual accomplishment record. 

Section 3; Synthesis 

Items of pupil information from a variety of sources m the five spheres 
of performance. In a variety of units of measurement, are combined so as 
to make sense to consumers, 
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Persons in Sphere V look for Information from persons in the other spheres 
on the elements of performance in Sphere V. information from^ol:her spheres 
is helpful when it is presented on the three phases of learning; for example, 
the number of times a verbal direction was repeated before the pupil began 
to perform the discrimination task used to assess the ability to process 
sensation. 



. Considering the pupils' abilities in all spheres of performance prevents 
overpromising in prediction. Considering the performance elements as the 
target of instructional program assists program evaluators and managers in 
designing and carrying out appropriate evaluation plans. 

Program impact is considered the aggregation of pupil change in performance 
elements related to movement in the three phases of learning in the spheres 
of performance* Performance element information can be aggregated or sampled 
along all the generally used demographic dimensions such as age , number of 
days of instruction, age of onset of condition, or type of instructional pro. 
gram. Performance element information can be used for study of relationships 
between school and nonschool situations. For example, latency: Is the pupil 
prompt for music class but late for work at the music store? The aim of 
education for deaf pupils may not be to make the deaf pupils hear. Rather, 
the aim of education for the deaf may be to assist the deaf pupils in learning 
to use any ability to process sound. One objective for a particular pupil 
might not be to increase the number of low-frequency sounds which can be 
consistently discriminated. Instead, the objective may be to increase the 
consistency with which the pupil communicates with hearing pupils in 
recreational situations. 

Conclusion 

The test of the Synthesis of Abilities Model is in the logic of its structure 
and in its utilit^y to the persons in the governance structure of education. 
Using the Synthesis of Abilities Model, '^nsumers of program evaluation can 
discuss the type of information they want and agree on the evidence they 
require. 

^ School staff and allied professionals can use the Synthesis of Abilities 
Model as a stimulant for organising their procedures for pupil study, docu- 
mentation, and reporting. Program evaluators can use the model as an organizer 
of pupil data for clustering, aggregation, and analysis and reporting. 



Among the primary users of the Synthesis of Abilities Model may well be the 
teachers and the parents.. It is teachers who take the responsibility for 
changing pupil performance through instruction. It is parents who have the 
responsibility for their children and youth during their period of growth 
and development. 

31 



23 



In the long run the chief beneficiaries will be P"Pil» "^he-elves. It 
Vb the pupils who will profit «>st from accurate and systenmtic infonwtion 
iJoii theJr «n abilities, for "as we think about ourselves, so do we become. 
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CHAPTER V 



Seminar Questions for Application of the Models 

This chapter has been included in response to the reconmendat ions of 
Project SEEM' 8 Operational Audit Team, The purpose of Chapter V is to 
raise questions in order to stimulate and facilitate reflection and 
discussion of the ideas presented in the document. Blank pages have been 
provided for noting additional ideas which occur to readers and discussants. 



The following questions have been derived from comments made by readers 
and reviewers of the document. The questions are intended to stimulate 
discussion, not to limit discussion. Individual readers and groups are 
encouraged to add their own questions, to revise those presented here, 
and to disagree with the ideas presented in the document. In preparing 
to respond to the questions, readers might consider the questions in two. . 
ways: first, in general; and second, in the particular program they have 
responsibility for. 



The questions have been ordered according to the sequence of the document. 
The questions have been displayed on each page so that they can be dupli- 
cated on 3 X 5 inch cards for ease in arrangement. 

Foreword 

1, Would more adequate program evaluation 
and pupil information assist special 
education decision makers to improve 
programs? 

2, Has the learner-centered emphasis long 
been recognized as a need in special 
education? 

3, Has the learner-centered emphasis been 
utilized as an excuse for not initiating 
program evaluation designs? 

4, Will a reader of this document find 
guidelines, procedures, and/or 
directives, as is typical of State 
Department of Education publications? 

5, Other 
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Chapter I, Introduction to Program Evaluation In Special gducation 



1. Is program evaluation a necessary part 
of ongoing planning and programming 
for exceptional individuals? 

2. Do special education personnel express 
a need for a document that would serve 
as a resource for developing evaluation 
designs? 

3. What is the importance of the phrase 
''agreeing to" in the definition of 
program evaluation? 

4. Is the current literature relating to 
evaluation of special' education programs 
acceptable? 

3. Should program evaluation be a program 
improvement device rather than a 
"proving" technique? 

6. Is there a relationship and interdependence 
between planning and evaluation? 

7. Do we have to define "program" before 
we can think about "program evaluation?" 

8. Would involving evaluators in program 
planning help special educators avoid 
past mistakes and insufficient utili- 
zation of resources? 

9. Do program evaluation requirements 
engender fearfulness and anxiety? 

10. Other 



Chapter II. The Consumer Model for Program Evaluation 

1. Why should an education program leader 
ask consumers questions about programs 
and program effects? 

2. Which elements in the governance 
structure have accepted the respon- 
sibility for program evaluation? 
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Is the pupil really the point of 
interaction of the consumers as 
depicted in the Consumer Model for 
Program Ev4.1uatlon? 

Do the actions of the Superintendent of Public 
Instruction directly or indirectly affect the pupilY 

Why are the courts placed next to the 
parents? 

Figure 1 omits mention of a national 
board of education. Why? 

What is the role of higher education 
in the Consumer Model for Program 
Evaluation? 

Have the essential features of the 
consumer model been identified? 

Does the pyramid building block 
structure appropriately describe 
the processes necessary to asiiign 
responsibility, provide resources, 
train people, and so on? 

Would it be more helpful to present 
a flow chart with appropriate feedback 
channels? 



4 . . 

Chapter III. Pupil Information in Special Education 

1, Is it good to be reminded that schools 
exist to enhance the performance o£^ 
pupils? 

2. Are variability and deviation in pupils 
the two most significant problems in 
the evaluation of special education 
programs? 

3» How helpful to teachers and parents are 
current measures of pupil performance? 

4. What useful structures for organizing 
pupil information are in existence now? 
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3. 

4. 
5. 
6. 
7. 

8. 
9. 

10. 
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Chapter IV, Synthesis of Abilities Model for Pupil Info rmation 



1. Is the vocabulary of the Synthesis 
of Abilities Model easily understood 
by all readers? 

2. What does "transmute" mean? 

3- Are the interrelationships among the 
elements identified in the Synthesis 
of Abilities ^iodel easily understood? 

4. What is this "internal environment" 
in Sphere I? 

5. Are the ten abilities stated in the 
simplest terms? (Easiest to understand, 
and still convey the intended meaning?) 

6. What are the relationships between and 
among the spheres of performance? 

7. Are the spheres of performance mutually 
exclusive? 

8. Does learning permeate all the spheres? 

9. What are examples of the preparation, 
acquisition, and use phases of learning? 

10. Are the spheres of performance 
hierarchical? 

11. Do special educators pay so much 
attention to how the pupil learns 
that they don't attend to what 
they're asking the pupil to learn? 

12. Where in the Synthesis of Abilities 
Model does data become information? 

13. Would using this model for pupil 
information lessen the amount of 
conclusion Jumping? 

14. Is the Synthesis of Abilities Model 
for Pupil Information an approach to 
evaluation design focussing on the 
learner? 

15. Other 
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Chapter V, Seminar Questions for Application of the Models 



!• Is the document a philosophical state- 
ment of evaluation as it relates to 
Special education? 

2. Is there general support for the 
document as a philosophical state** 
ment of evaluation as it relates to 
special education? 

3. Does the document provide a theoretical 
framework that will stimulate the 
development of operational evaluation 
designs? 

4. Is there general consensus and 
enthusiasm for the concept of a 
consumer model for program evaluation 
with the focus on the learner? 

3. What does all this mean to me as a 

__? (Fill in your 

own choice of sel f -descriptors ,) 

6. Should both models, the "Consumer 
Model for Program Evaluation" and 
the "Synthesis of Abilities Model 

for Pupil Information," be considered 
in the design of comprehensive evalu* 
ation systems? 

7. Do these models apply to all education 
programs and all pupils? 

8. Should ongoing development of the 
models be carried on? 

9. Would essential activities to carry on 
ongoing development of the models 
include. • • • 

• opportunities for field review? 

• revision? 

• development of operational manuals/ 
handbooks and/or guidelines? 

• testing? 

• pilot implementation? 

10. Would a pilot endeavor to apply the 
theoretical model to programs in 
operation be helpful in identifying 
the process and procedures necessary 
to operational ize the model?' 
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11. Should the cost of implementing the 
evaluation models at the operating 
level be considered as a major 
component of any field test design? 

12. Can discussing these models help a 
school staff create their own 
adaptation, tailored to fit their 
pupils, programs, and consumers? 

13. What would be the effect of incorporating 
specific concrete examples into the 
document? 

14. What additional costs would be incurred 
as a result of adopting these models? 

15. Are there costs presently incurred 
which would not be necessary if these 
models were adopted? 

16. What would be the effect of h'inging 
together small groups of perse • to 
discuss and critique the doer lu^ut? 

17. Is the seminar approach an appropriate 
way to become familiar with the content 
and intent of the document? 

18. Is there a role for institutions of 
higher education in these models? 
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ABSTRACT 



This document presents two new models for educational evaluation with 
emphasis on special education: the Consumer Model for Program Evaluation 
and the Synthesis of Abilities Model for Pupil Information. 

The Consumer Model for Program Evaluation covers the governance structure 
of education and emphasizes the pupil as the focal point. The chief 
features of the Consumer Model are responsibility, budget? trained 
people, reporting, use of information, design, measures, and basis for 
evaluation* 



The Synthesis of Abilities Model for Pupil Information is applicable to 
all pupils in all learning situations. The chief features of the Synthesis 
of Abilities Model are abilities, spheres of response, analyses, and 
synthesis. Ten abilities are identified in five spheres of performance. 
Quantitative and contrastive analyses are discussed. Emphasis is placed 
on pupil performance elements in three phases of learning: preparation, 
acquisition, and use. Learning is defined as a measurable, verifiable 
change in performance. 



Program evaluators and managers at all levels of the governance structure 
of education are encouraged to work with consumers to clarify their infor- 
mation needs, specify the evidence required, and provide the resources 
necessary to supply accurate, timely, and helpful information. 
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